Photogrammetry is the science of obtaining measurements from photographs, especially for recovering the exact or nearly-exact positions of surface points. While photogrammetry is emerging as a robust, non-contact technique to obtain measurements of objects, scenes, landscapes, etc., there are limitations to existing methods, some of which, for example, are set forth in the following few paragraphs.
Accurate three-dimensional (3D) digital representations of objects can be obtained using methods that utilize active-sensing techniques, such as systems that emit structured light, laser beams or the like, record images of objects illuminated by the emitted light, and then determine the 3D measurements from the recorded images. A laser scanner is an example of a standalone device that utilizes structured light to generate measurements of objects. When used in mobile devices, such as smartphones and tablets, emission of the structured light used for 2D and 3D image generation can be achieved by including a separate hardware device as a peripheral. This peripheral is configured to emit, for example, structured light to generate a point cloud (or depth map) from which data about the object of interest can be derived using photogrammetric algorithms. Use of such a peripheral device to provide active sensing methods are provided by, for example, Structure Sensor (see the internet URL structure.io), the DPI-8 kit or the DPI-8SR kit products (see the internet URL www.dotproduct3d.com). While often providing accurate image data, it is nonetheless cumbersome for users to have to add a clamp-on or other type of peripheral equipment to their mobile devices. Alternatively, active sensing means can be integrated into mobile devices, such as in Google's Tango® product.
Existing passive photogrammetry methods—that is, methods that do not use structured light, lasers or the like but which, for example, utilize images captured by a camera from which to derive measurements, etc. can also be problematic to use. Conventional stereo/2D or 3D cameras typically obtain two images of an object simultaneously from two viewpoints that are typically separated, for example, by the interpupillary distance (IPD) of a person (which can range from about 52 to about 78 mm according to the 1988 Gordan et al. “Anthropometric Survey of US Army Personnel, Methods and Summary Statistics.” TR-89-044. Natick Mass.: U.S. Army Natick Research, Development and Engineering Center). Such stereo images generally have insufficient parallax for high-quality measurement when used to obtain data regarding distant objects (e.g., objects more than a few (about one to about five) meters away from the cameras). To obtain suitable parallax using such methods, the user will be directed to use a template or framework incorporated in, for example, software associated with the image-capture device to guide orientation of the image-capture device relative to the object of interest. This technique can ensure that a sufficient number of appropriately overlapping images of the object of interest are obtained. Alternatively, the user can be provided with general instructions of how to orient the camera and/or object so as to obtain appropriate overlap. Both of these techniques for guiding the user can be used to provide accurate visualization of the object of interest but are nonetheless cumbersome and prone to user error.
It is possible to obtain accurate measurements from photographs by using multiple images of an object of interest. When placed in a 3D context (i.e., “multiple view geometry”), the three-dimensional points from an object of interest can be estimated from measurements from two or more photographic images taken from different positions. Corresponding points are identified on each image. A line of sight (or ray) can be constructed from the camera location to the point on the object. Triangulation allows determination of the 3D location of the point both in relation to the object's orientation in space, as well as with regard to that point's orientation and/or position in relation to other points.
Methods for passive photogrammetry where 3D digital representations of the object(s) of interest can be used to derive measurements and other detail of interest are disclosed in U.S. Pat. No. 8,897,539 titled “Using images to create measurements of structures through the videogrammetric process” and PCT Publication No. WO2013/173383 by Brilakis et al. titled “Methods and apparatus for processing image streams,” U.S. Pat. No. 8,855,406 to Lim, et al. titled “Egomotion using assorted features,” the disclosures of which are incorporated in their entireties by this reference. Notably, the methodologies disclosed in each of these references require the use of two cameras to capture 2D images from which a 3D digital representation can thereby be obtained.
An example of fairly accurate passive photogrammetry that utilizes multiple images generated from a single camera is provided by Photomodeler (photomodeler.com). This software product allows a user to generate a 3D digital representation of an object of interest from multiple overlapping images, where the relevant detail is provided by the orientation of images in a known area of space. In some implementations, accurate measurements can be obtained from the 3D digital representations of the object(s) of interest. However, Photomodeler requires a user to conduct explicit calibration that occurs in a separate step to achieve such accuracy. Once the 3D orientation is obtained, measurement and other detail information regarding the object of interest can be provided for use. At least part of this calibration step comprises users perform a manual boundary identification. This calibration process is time consuming, currently requiring the user to generate a chessboard marker comprising a minimum number of images taken from different angles and distances with respect to the image-capture device, whereby more images will provide more accurate calibration. Moreover, to measure objects of interest that are longer distances from the camera, accurate measurements of the object of interest require larger calibration surface (e.g., about 6 ft.×about 6 ft. (1.82 meters by 1.82 meters)). As might be recognized, this physical calibration step provides the information necessary to orient the object(s) of interest in space so as to make it possible to provide 3D digital representations of the object(s) of interest thereof so that measurements can be obtained.
Recently issued U.S. Pat. No. 8,953,024, the disclosure of which is incorporated herein in its entirety, indicates that 3D digital models of scenes can be generated using a passive digital video camera using, in one implementation, structure from motion algorithms. Among other things, there is no disclosure in the '024 patent that sufficient detail about individual objects present in the scene can be obtained to allow specific parameters of such objects to be resolved in order to obtain accurate 3D digital representations suitable to provide measurements or the like.
In light of these and other issues, there remains a need for improvements in photogrammetry that allow a user to obtain accurate 3D digital representations of an object of interest (or a collection of objects of interest) without the need for use of two camera image acquisition and/or the use of cumbersome processing steps. Still further, it would be desirable to have method and devices to obtain accurate 3D digital representations of the object(s) using a single image-capture device, such as those integrated into mobile devices (e.g., smart phones, tablets, etc.). Yet further, it would be desirable to be able to obtain substantially accurate measurements of object(s) of interest in a scene. The present invention provides this and other benefits.